Papers with computational cost challenge
MoSLD: An Extremely Parameter-Efficient Mixture-of-Shared LoRAs for Multi-Task Learning (2025.coling-main)
Copied to clipboard
| Challenge: | LoRA is a key technique for fine-tuning large pre-trained models, yet its performance in multi-task learning scenarios often falls short. |
| Approach: | They propose a mixture-of-shared-LoRAs model with a dropout strategy . they propose to share the upper projection matrix among different experts . |
| Outcome: | The proposed model exhibits excellent performance in both single-task and multi-task scenarios with robust out-of-domain generalization capabilities. |